Performance Results

These are the measurements captured while developing this PR. The smoke result is from the current pushed commit (426ea33); the larger scale artifacts were captured during the same PR work before the final no-globals cleanup, so they are useful as scale evidence but should be rerun if we need final apples-to-apples numbers on the exact PR commit.

Current fused actor smoke

Command:

MODAL_PROFILE=everett-38139 uv run --extra models modal run pipelines/transcribe_diarize/modal_app.py::measure \
  --source '/audio/flac/Build_Scalable_Batch_Inference_Pipelines_in_3_Lines_Daft_GPT_vLLM_0.00_2.38.flac' \
  --config parakeet+sortformer

Result:

• First service call wall: 31.0s
• Measured collect wall after warmup: 0.2s
• Fused actor status: model_residency: TranscribeDiarizeVad.__init__
• Warm fused process: 0.1704s on the 2.38s clip
• Warm ASR stage: 0.0819s; warm diarization stage: 0.0565s
• Output shape: 2 ASR segments, 1 speaker segment

This smoke is mainly a runtime/API validation because the clip is too small for stable throughput conclusions.

2-hour L40S scale run

Artifact: .context/transcribe_diarize/MODE_COMPARE_sports_basement_2h_L40S_linear_vad.json

• Source: /audio/flac/sports_basement_2h.flac
• Audio duration: 7200.0s
• Wall time: 193.934s
• Throughput: 37.13x realtime
• Peak VRAM: 11.998 GB

The old linear path did not produce a comparable throughput number:

• Daft linear comparison failed after 12.01s with Need at least 1 series to perform concat.
• Direct whole-file compacted ASR diagnostic OOMed on the same 2h input after MarbleNet compaction: 5575.68s model audio, attempted 289.54 GiB allocation on an L40S (44.39 GiB).

Artifact: .context/transcribe_diarize/DIAG_LINEAR_ASR_2H_FULL.json

Real-file processing runs

IMG_3800:

• Source: /Users/everettkleven/Desktop/IMG_3800.mov
• Audio duration: 9377.19s (2:36:17)
• Client wall: 259.35s
• Throughput: 36.16x realtime
• ASR segments: 2025
• Word timestamps: 24967
• Speaker turns: 1631
• VAD removed: 974.20s

Artifacts:

• .context/transcribe_diarize/IMG_3800_transcribe_summary.json
• .context/transcribe_diarize/IMG_3800_report_summary.json

Zoom batch:

• Source glob: /audio/flac/zoom_2026-06-*.flac
• Files: 4
• Total audio duration: 11616.21s (3:13:36)
• Client wall: 268.48s
• Throughput: 43.27x realtime
• ASR segments: 2785
• Speaker turns: 2932
• VAD removed: 3464.03s

Artifact: .context/transcribe_diarize/zoom_marketing_launch_batch_transcribe_summary.json

MarbleNet VAD threshold bench

Artifact: .context/transcribe_diarize/VAD_BENCH_smoke.json

On the 155.64s sample, threshold 0.7 was selected by the current heuristic:

• Removed: 32.40s (20.82%)
• WER delta vs no VAD: 0.0445
• Deletion-WER: 0.0157
• Reason: largest seconds removed while staying within WER <= 0.050 and deletion-WER <= 0.020

Threshold 0.8 removed slightly more (33.68s) but exceeded the WER tolerance (0.0524).

Profiler smoke

Artifact: .context/transcribe_diarize/transcribe_diarize_profile_smoke_current.summary.txt

• Self CPU total: 2.130s
• Self CUDA total: 652.885ms
• Top CUDA kernel: ampere_sgemm_128x32_tn, 181.528ms (27.80% self CUDA)